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REAL PARTY IN INTEREST 

The assignee of the entire right, title, and interest of the above-identified appHcation, 
Clever Sys, Inc., is the real party in interest in the application. 

RELATED APPEALS AND INTERFERENCES 

There are no other appeals and/or interferences related to the above-identified 
application. 

STATUS OF CLAIMS 

Claims 1-10, 12, 14-17, and 19-55 have been finally rejected. Claims 11, 13, and 18 are 
allowed. Claims 1-10, 12, 14-17, and 19-55 are on appeal 

STATUS OF AMENDMENTS 
Appellants have filed no amendments after the mailing date of the final Office Action. 

SUMMARY OF CLAIMED SUBJECT MATTER 

Independent claim 1 

Appellants' invention is directed to a video-based animal behavior analysis system 
including a computer configured to determine a position and shape of an animal fi*om video 
images and to characterize activity of the animal as one of a set of predetermined behaviors 
based on an analysis of changes in the position and the shape over time. See Specification, pg. 
15, line 23 - pg. 16, line 4. In one embodiment, the computer 150 receives video images of an 
animal and processes the video images to segregate the animal from the background images, 
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tracks the animal over several frames, classifies the shape and posture of the animal, and 
characterizes the activity of the animal as one of a set of predetermined behaviors. See 
Specification, pg. 19, line 12 - pg. 20, line 2; FIG. 2. In one embodiment, the set of 
predetermined behaviors includes sleeping, eating, drinking, walking, and running. See 
Specification, pg. 21, lines 7-10. 

Independent claim 8 

Appellants' invention is directed to a method of determining and characterizing activity 
of an animal using computer processing of video images including detecting an animal in the 
video images, tracking changes in position and shape of the animal over a pluraUty of the video 
images, classifying the changes in position and shape of the animal as postures, and 
characterizing activity of the animal as one of a set of predetermined behaviors based on a 
comparison of a sequence of the postures to pre-trained models or rules of the set of 
predetermined behaviors. See Specification, pg. 20, line 3 - pg. 21, line 10; FIG. 3. In one 
embodiment, the set of predetermined behaviors includes sleeping, eating, drinking, walking, 
and running. See Specification, pg. 21, lines 7-10. 

GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

1. The rejection of claims 1, 6, and 7 as unpatentable under 35 U.S.C. § 103(a) over 
U.S. Patent No. 6,072,903 to Maki et al. ("Maki") in view of U.S. Patent No. 4,888,703 to Baba 
et al C'Baba")^ 
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2. The rejection of claims 2-5 as xmpatentable under 35 U.S.C. § 103(a) over Maki 
in view of Baba and further in view of U.S. Patent No. 6,715,444 to Yabusaki et al 
("Yabusaki"). 

3. The rejection of claims 8-10, 12, 14-17, 19, and 21-55 as unpatentable under 35 
U.S.C. §103(a) over Maki in view of Baba and further in view of U.S. Patent No. 6,242,456 to 
Shuster et al. ("Shuster")* 

4. The rejection of claim 20 as unpatentable under 35 U.S.C. § 103(a) over Maki in 
view of Baba and further in view of U.S. Patent No. 5,870,138 to Smith et al. ("Smith"). 

ARGUMENT 

To establish a prima facie case of obviousness mider § 103(a), three basic criteria must 
be met. First, there must be some suggestion or motivation, either in the references themselves 
or in the knowledge generally available to one or ordinary skill in the art, to modify the 
reference or to combine the reference teachings. Second there must be a reasonable expectation 
of success. Finally, the prior art reference (or references when combines) must teach or suggest 
all the claim limitations. The teaching or suggestion to make the claimed combination and the 
reasonable expectation of success must both be found in the prior art, not in appHcant's 
disclosure. In re Vaeck 947 F.2d 488, 20 U.S.P.Q.2d 1438 (Fed. Cir. 1991). 

1. Rejection of Claims 1, 6, and 7 

The Examiner maintains the rejection of claims 1, 6, and 7 under U.S.C. § 103(a) as 
being unpatentable over Maki in view of Baba. Appellants respectfully disagree with the 
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Examiner's reading of the disclosures in Maki and Baba and submit that these references do 
not render the subject matter of claims 1, 6, and 7 obvious under 35 U.S.C. § 103(a). 

Maki discloses an image processing system and method for head-tracking (following the 
movement of a head of a person), video compression capable of decreasing the data amount 
required for image communication by extracting the motion vector of the person in a 
teleconference, and three-dimensional pointing in a virtual reality system. Maki, col. 1, lines 11- 
1 8. A second embodiment of the system of Maki creates a surface model of the head of a 
person. Maki, col. 14, lines 52-53. This second embodiment includes a 3-D motion information 
extraction unit that determines the position and posture of the head of the person, and a distance 
information detector that detects information on the distance to the object and reconstructs the 
shape according to the position and posture of the head. Maki, coL 15, lines 9-16; 49-51. A 
third embodiment of the system of Maki obtains images of an object using a video camera, 
analyzes the images for use in three-dimensional pointing, and acquires and tracks the posture of 
the object. Maki, col. 23, line 64 - col. 24, line 11. The system of Maki detects the position and 
posture of a person's head for the purposes of creating a model of the person's head or using the 
position and posture of the head as a three-dimensional pointer. Maki, col. 2, lines 47-59. 

Claim 1 recites "a computer configured to determine a position and shape of an animal 
from video images and to characterize activity of said animal as one of a set of predetermined 
behaviors based on an analysis of changes in said position and said shape over time." Maki does 
not teach or disclose a computer configured "to characterize activity of said animal as one of a 
set of predetermined behaviors" as recited in claim 1 . There is no teaching or disclosure in Maki 
of using changes in the position and shape of an animal to characterize activity of an animal, 
much less characterize activity of an animal as one of a set of predetermined behaviors. The 
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section of Maki cited by the Examiner (Col. 15, lines 49 - 57) describes the distance information 
detector which merely gathers distance information to use to reconstruct the object. Further, 
claim 2 of Maki (Col. 32, lines 56-67) referred to by the Examiner, which is dependent from 
claim 1 of Maki, recites additional information about the distance information detector for use in 
acquiring the shape of the object as set forth in the preamble of claim 1 of Maki. The Examiner 
also cites to Col. 24, line 63 - CoL 25, line 1 1 of Maki, which discloses that feature points are 
used to track the object over a series of images. 

There is no teaching or disclosure in Maki of a computer "to characterize activity" of an 
animal, much less characterizing the activity of the animal as one of a set of predetermined 
behaviors. Maki teaches identifying feature points from a series of moving images of the head of 
a person, and tracking the motion of the head over the series of moving images. Tracking the 
motion of a person's head is not "characterizing the activity" of the person. 

The Examiner stated that Maki does not explicitly state "one of a set of predetermined 
behaviors based on an analysis of changes in position or shape," and that Baba teaches a 
processor for processing an image taken by a camera to observe the behavior of a fish and to 
produce an alarm signal when the abnormal behavior of the fish is observed, for predicting 
behavior. 

Baba discloses an apparatus for monitoring the toxicant contamination of water by using 
aquatic animals. The Baba apparatus receives an image of the silhouette of an aquatic animal, 
such as a fish, and then calculates the position of the center of gravity of the fish image. Baba, 
coL 8, lines 24-31, 64-66. The Baba apparatus then determines the position of the center of 
gravity along a vertical axis and uses that vertical position information to produce a normaUzed 
distribution of frequencies versus the vertical position of the center of gravity. Baba, col. 9, line 
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24 - col. 10, line 25. This distribution is used to determine whether the behavior of the fish is 
normal. In another embodiment, the Baba apparatus determines the speed of the fish by 
calculating the rate of change of the position of the center of gravity of the fish. Baba, col 10, 
line 45 - col. 1 1 , line 49. The Baba apparatus may also determine the amount of movement of 
the fins of the fish using the same methods used to determine the movement of the whole fish, 
i.e., by using position information of the center of gravity. Baba, col. 14, lines 49-61. In another 
embodiment, the Baba apparatus detects the posture of the fish, again by first determining the 
center of gravity of the fish. The Baba apparatus may also reduce the image of the fish to a line 
to determine the angle of the fish image with respect to the surface of the water. Baba, col. 15, 
line 6-61. 

All of the embodiments of Baba reduce the image of the fish to either a single point 
(center of gravity) or a single line. This one-dimensional information is then fiirther processed to 
make a determination of whether the behavior is normal or abnormal. 

Claim 1 recites a computer configured "to characterize activity of said animal as one of a 
set of predetermined behaviors based on an analysis of changes in said position and said shape 
over time." An analysis of changes in the shape of an animal necessarily requires information in 
at least two dimensions. In Baba, the apparatus reduces the fish image to the center of gravity 
(i.e., a single point) of the fish or a single line segment, and uses this point or line to determine 
whether the behavior of the fish is normal or abnormal. Baba does not teach or disclose 
analyzing changes in the shape of the animal over time. Tracking the position of a single point 
or single line that represents the fish does not teach an analysis of changes in the shape of an 
animal over time. A single point or a single line does not provide any information about the 
shape of an animal. In fact, Baba does not take the shape of the animal into account at all: 
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"[T]he recognition of the aquatic animal is not influenced by any factors, such as , , . the postural 
change during the movement of the aquatic animal" Baba, col 2, lines 56-60. Thus, Baba does 
not teach or disclose a computer configured "to characterize activity of said animal as one of a 
set of predetermined behaviors based on an analysis of changes in said position and said shape 
over time" as recited in claim 1 . 

Further, there is no motivation to combine Maki with Baba, and the Examiner has not 
pointed to any suggestion that one skilled in the art should combine them. Maki discloses an 
image processing system and method for head-tracking (following the movement of a head of a 
person), video compression capable of decreasing the data amount required for image 
communication by extracting the motion vector of the person in a teleconference, and three- 
dimensional pointing in a virtual reality system. Maki, col. 1, lines 11-18. Baba discloses an 
apparatus used to monitor the toxin levels in water using aquatic animals. Maki and Baba are 
directed to completely non-analogous arts. One of ordinary skill in the art of image processing 
systems would have no motivation to combine a reference relating to image processing systems 
with a reference relating to monitoring the level of toxicity of water, and one of ordinary skill in 
the art of monitoring the toxicity of water would have no motivation to combine a reference 
relating to monitoring the toxicity of water with a reference relating to image processing 
systems. It is of course impermissible hindsight reasoning to rely on the current application to 
find a motivation to combine. 

None of the cited references, either alone or in combination, teaches or discloses all of 
the limitations of claim 1, and there is no motivation to combine the references. Applicants 
respectfully submit that claim 1 is not obvious in view of Maki and Baba and is allowable. 
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Claims 2-7 depend, directly or indirectly, from claim 1 and are therefore allowable for at least 
the same reasons. 

Regarding claims 6 and 7, the Examiner unfortunately persists in his erroneous assertion 
that Maki discloses a biological mouse and a biological rat without explanation of his reasons for 
making such a conclusion. The portion of Maki cited by the Examiner (Col. 2 , lines 20-26) 
discloses "a three-dimensional mouse enabling movement and a pointing action in a three- 
dimensional space by the operation of the buttons on the device.''^ (emphasis added). It is clear 
from this quotation that the mouse of Maki is a device with buttons^ i.e. a computer mouse. 
Neither a biological mouse nor a biological rat is a device that enables a pointing action by 
operation of its buttons. None of the cited references teaches or discloses all of the limitation of 
claims 6 and 7, and there is no motivation to combine the references. Applicants respectfully 
submit that claims 6 and 7 are not obvious and are allowable. 

2. Rejection of Claims 2-5 

The Examiner maintains the rejection of claims 2-5 under U.S.C. § 103(a) as being 
unpatentable over Maki in view of Baba, and further in view of Yabusaki. Appellants 
respectfully disagree with the Examiner's reading of the disclosures in Maki, Baba, and 
Yabusaki and submit that these references do not render the subject matter of claims 2-5 
obvious under 35 U.S.C. § 103(a). 

As set forth above, Maki in combination with Baba does not disclose all of the limitations 
of independent claim 1 and tliere is no motivation to combine the two references, and for those 
reasons alone the rejection should be withdrawn. 
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Regarding claim 2 and its dependent claims, even if one were to presume that the 
combination of Maki and Baba was proper, Yabusaki does not disclose a "video digitization unit 
. . . for . . . converting said video images from analog to digital format." The portion of Yabusaki 
cited by the Examiner (Col 3, lines 38-54) discloses that a vibration of a cage is detected as an 
electrical signal by the displacement of a sensor, and that this electrical signal is detected by a 
frequency analyzer and converted into a digital signal An electrical signal created by 
displacement of a vibration sensor that is converted into a digital signal does not teach or 
disclose a video digitization unit that converts video images from analog to digital format. 
Further, there in no motivation to combine Yabusaki with either Maki or Baba, as Yabusaki has 
nothing to do with processing images or determining the toxicity of water. None of the cited 
references, either alone or in combination, teaches or discloses all of the limitations of claim 2, 
and there is no motivation to combine the references. Appellants respectftiUy submit that claim 2 
is not obvious and is allowable. 

Regarding claim 3 and its dependent claims, Maki does not disclose "an animal 
identification, segregation, and tracking module." The portion of Maki cited by the Examiner 
(Col 29, lines 23-28) discloses an image calculator to determine the luminance of the feature 
points on the object. None of the cited references, either alone or in combination, teaches or 
discloses all of the limitations of claim 3, and there is no motivation to combine the references. 
Appellants respectfully submit that claim 3 is not obvious and is allowable. 

Regarding claim 4 and its dependent claims, Maki does not disclose "a behavior 
identification module for characterizing activity of said animal." The portion of the specification 
of Maki cited by the Examiner (Col. 23, lines 37-51) discloses that the three-dimensional 
position and posture of the object is determined using feature points. Claim 3 of Maki cited by 
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the Examiner discloses a method of acquiring a shape of an object. Neither of these citations 
discloses the "behavior identification module" as recited in claim 4, None of the cited 
references, either alone or in combination, teaches or discloses all of the limitations of claim 4, 
and there is no motivation to combine the references. Appellants respectfully submit that claim 4 
is not obvious and is allowable. 

Regarding claim 5, even if one were to presume that the combination of Maki and Baba 
was proper, Maki does not disclose a "standard animal behavior storage module that stores 
information about known behavior of a predetermined standard animal" Claim 4 of Maki cited 
by the Examiner discloses a method of estimating distance information on an object as part of the 
method of acquiring a shape of an object of claim 3 of Maki. The portion of the specification of 
Maki cited by the Examiner (Col 7, lines 19-33) discloses a memory for storing time-series 
images; a series of moving pictures. Neither of these citations discloses a "standard animal 
behavior storage module that stores information about known behavior of a predetermined 
standard animal." A series of moving pictures is not information about known behavior of a 
predetermined standard animal None of the cited references, either alone or in combination, 
teaches or discloses all of the limitations of claim 5, and there is no motivation to combine the 
references. Appellants respectfully submit that claim 5 is not obvious and is allowable. 

3. Rejection of Claims 8-10, 12, 14-17, 19, and 21-55 

The Examiner maintains the rejection of claims 8-10, 12, 14-17, 19, and 21-55 under 
U.S.C. § 103(a) as being unpatentable over Maki in view of Baba, and further in view of 
Shuster. Appellants respectfully disagree with the Examiner's reading of the disclosures in 
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Maki, Baba, and Shuster and submit that these references do not render the subject matter of 
claims 8-10, 12, 14-17, 19, and 21-55 obvious under 35 U.S.C. § 103(a). 

Claim 8 recites "classifying said changes in position and shape of said animal as postures; 
and characterizing activity of said animal as one of a set of predetermined behaviors based on a 
comparison of a sequence of said postures to pre-trained models or rules of said set of 
predetermined behaviors." As set forth above regarding claim 1, Maki does not disclose 
characterizing activity of an animal as one of a set of predetermined behaviors. Maki also does 
not disclose comparing a sequence of postures to pre-trained models or rules of a set of 
predetermined behaviors. The portion of Maki cited by the Examiner (Col. 27, lines 20-3 1) 
discloses a comparison section that generates synthesized images and evaluates postures 
according to the similarity of the generated images to the image of the object. In other words, 
Maki teaches comparing images to determine postures, not "comparing a sequence of postures to 
pre-trained models or rules of a set of predetermined behaviors" as recited in claim 8. 

The Examiner stated that neither Maki nor Baba teaches classifying changes in position 
and shape of the animal as postures, but that Shuster teaches this limitation. The portion of 
Shuster cited by the Examiner (Col, 3, lines 33-62) Hsts various compulsive or stereotypical 
behaviors in dogs, and states that these behaviors can also be observed in other animals. Shuster 
merely provides a list of various behaviors, and does not teach or disclose "tracking changes" 
and "classifying said changes in position and shape of said animal as postures" as recited in 
claim 8. 

As set forth above, there is no motivation for combining Maki with Baba. Further, there 
is no motivation to combine Shuster with either Maki or Baba, and the Examiner has not pointed 
to any suggestion that one skilled in the art should combine them. Maki discloses an image 
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processing system and method for head-tracking (following the movement of a head of a 
person), video compression capable of decreasing the data amount required for image 
communication by extracting the motion vector of the person in a teleconference, and three- 
dimensional pointing in a virtual reality system. Maki, col. 1, lines 11-18. Shuster discloses a 
method for treating a repetitive behavior disorder in animals by administering doses of one or 
more NMD A receptor antagonists. Shuster, col. 1, lines 56-61. These two references are 
directed to completely non-analogous arts. Applicants respectfully submit that neither Maki nor 
Shuster provide any possible motivation to combine the image processing system of Maki with 
the method of treating a disorder in animals with a certain type of drug of Shuster. It is of course 
impermissible hindsight reasoning to rely on the current appUcation to find a motivation to 
combine. 

Baba discloses an apparatus for determining the toxicity of water. Shuster discloses a 
method for treating a repetitive behavior disorder in animals by administering doses of one or 
more NMDA receptor antagonists. These two references are directed to completely non- 
analogous arts. AppUcants respectfully submit that neither Baba nor Shuster provide any 
possible motivation to combine the apparatus of Baba with the method of treating a disorder in 
animals with a certain type of drug of Shuster. It is of course impermissible hindsight reasoning 
to rely on the current application to find a motivation to combine. 

None of the cited references, either alone or in combination, teaches or discloses all of 
the limitations of claim 8, and there is no motivation to combine the references. Appellants 
respectfully submit that claim 8 is not obvious and is allowable. Claims 9, 10, 12, 14-17, and 19- 
55 depend, directly or indirectly, fi^om claim 8 and are therefore allowable for at least the same 
reasons. 
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Regarding claim 9 and its dependent claims, Maki does not disclose "describing said 
sequence of said postures as behavior primitives and aggregating said behavior primitives as into 
actual behavior over a range of images." The portion of the specification of Maki cited by the 
Examiner (Col. 22, line 56 - CoL 23, line 7) discloses determining a distance image by executing 
a certain evaluation function. Claim 1 of Maki cited by the Examiner discloses a method for 
acquiring the shape of a target object. Neither determining a distance image nor acquiring a 
shape of an object teach "describing said sequence of said postures as behavior primitives and 
aggregating said behavior primitives as into actual behavior over a range of images" as recited in 
claim 9. None of the cited references, either alone or in combination, teaches or discloses all of 
the Umitations of claim 9, and there is no motivation to combine the references. Appellants 
respectfully submit that claim 9 is not obvious and is allowable. 

Regarding claim 10 and its dependent claims, Maki does not disclose "describing a set of 
conditions and rules required for characterizing said activity; and matching and testing generated 
features to see if said conditions and rules are satisfied." The portion of Maki cited by the 
Examiner (CoL 4, lines 18-27) discloses an estimation section for estimating information on the 
distance to a target object. An estimation section for estimating distance to a target object does 
not teach or disclose "describing a set of conditions and rules required for characterizing said 
activity; and matching and testing generated features to see if said conditions and rules are 
satisfied" as recited in claim 10. None of the cited references, either alone or in combination, 
teaches or discloses all of the limitations of claim 10, and there is no motivation to combine the 
references. Appellants respectfully submit that claim 10 is not obvious and is allowable. 

Regarding claim 12 and its dependent claims, Maki does not disclose classifying changes 
in position and shape of an animal as postures "using statistical and contour-based shape 
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information." The portion of Maki cited by the Examiner (Col. 9, Unes 14-27) discloses 
extracting connected feature points by finding where contours in an image intersect each other. 
Extracting connected feature points from an image does not disclose classifying changes in 
position and shape of an animal as postures "using statistical and contour-based shape 
information" as recited in claim 12. None of the cited references, either alone or in combination, 
teaches or discloses all of the elements of claim 12, and there is no motivation to combine the 
references. Appellants respectfully submit that claim 12 is not obvious and is allowable. 

Regarding claim 14, Maki does not disclose classifying changes in position and shape of 
an animal as postures using "contour-based shape information selected from the group consisting 
of curvature measures, thickness measures, relative orientation measures, length measures, and 
comer points." The portion of Maki cited by the Examiner (Col. 9, lines 14-23) discloses 
extracting connected feature points by finding where contours in an image intersect each other. 
Extracting connected feature points does not teach a way of classifying changes in position and 
shape of an animal as postures as recited in claim 14. None of the cited references, either alone 
or in combination, teaches or discloses all of the limitations of claim 14, and there is no 
motivation to combine the references. Appellants respectfully submit that claim 14 is not 
obvious and is allowable. 

Regarding claims 16, 17, 19, 21, 28, 33, 39 and 48, none of the portions of Maki cited by 
the Examiner correspond to the claimed limitations. Appellants respectfully submit that each of 
these claims is not obvious and is allowable. 

The Examiner's grouping of claims 15, 22-27, and 29-32, and 34-38, 40-47, and 49-55 
and rejecting them because analogous arguments apply does not enable Appellants to properly 
respond to the rejection and hence the rejection should be withdrawn. Quite clearly, none of the 
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limitations of these claims are addressed by Maki or Baba, and Shuster fails to supply the 
teaching or suggestions that would render these claims obvious. For example, claim 15 recites a 
set of model postures, none of which are disclosed in any of the cited references. 

4. Rejection of Claim 20 

The Examiner maintains the rejection of claim 20 under U.S.C. § 103(a) as being 
unpatentable over Maki in view of Baba, and further in view of Smith. Appellants respectfully 
disagree with the Examiner's reading of the disclosures in Maki, Baba, and Smith and submit 
that these references do not render the subject matter of claim 20 obvious under 35 U.S.C. § 
103(a). 

As set forth above, neither Maki nor Baba, either alone or in combination, teaches or 
discloses all of the limitations of claim 8, the independent claim from which claim 20 indirectly 
depends, and does not teach or disclose all of the limitations of the intervening claims. Thus, 
claim 20 is allowable. 

Further, there is no motivation to combine Smith with either Maki or Baba, and the 
Examiner does not provide one. The portion of Smith cited by the Examiner (Col. 17, lines 41- 
55) discloses a device that analyzes output of a face changer to output the probability of input 
images belonging to a specified set of expressions, and that the device may be realized as a 
Hidden Markov Model There is no suggestion or motivation in Maki to use a Hidden Markov 
Model to output the probability of input images belonging to a specified set of expressions in 
Maki's system. Indeed, to do so would not appear to be possible and still use the algorithms and 
methods disclosed in Maki. Also, there is no motivation to combine Smith, a reference that 
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discloses a device that analyzes output of a face changer, with Baba, a reference that discloses 
apparatus used for determining the toxicity of water. 

None of the cited references, alone or in combination, teach or disclose all of the 
limitations of claim 20, and there is no motivation to combine the references. Appellants 
respectfully submit that claim 20 is not obvious and is allowable. 

CONCLUSION 

For the foregoing reasons, it is respectfully submitted that Appellants' claims are not 
rendered obvious by and are, therefore, patentable over the art of record, and the Examiner's 
rejections should be reversed. 



Respectfully submitted, 



Yiqing Liang et al 





Wendi R. Schepler (Reg. No. 43,091) 
WHITE & CASE LLP 

11 55 Avenue of the Americas 
New York, NY 10036 
(650)213-0323 
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CLAIMS APPENDIX 



1 . A video-based animal behavior analysis system, comprising: 

a computer configured to determine a position and shape of an animal firom video images 
and to characterize activity of said animal as one of a set of predetermined 
behaviors based on an analysis of changes in said position and said shape over 
time. 

2. The system of claim 1 , further comprising: 

a video camera and a video digitization unit coupled to said computer for capturing said 
video images and converting said video images from analog to digital format. 

3. The system of claim 2, fiirther comprising: 

an animal identification, segregation, and tracking module receiving said video images. 

4. The system of claim 3, wherein said computer further includes a behavior identification 
module for characterizing activity of said animal, said behavior identification module being 
coupled to said animal identification, segregation, and tracking module. 

5. The system of claim 4, wherein said computer further includes a standard animal 
behavior storage module that stores information about known behavior of a predetermined 
standard animal for comparing the activity of said animal, said standard animal behavior storage 
module being coupled to said behavior identification module. 

6. The system of claim 1 , wherein said animal is a mouse. 
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7. 



The system of claim 1, wherein said animal is a rat. 



8. A method of determining and characterizing activity of an animal using computer 
processing of video images, comprising the steps of: 

detecting an animal in said video images; 

tracking changes in position and shape of said animal over a plurality of said video 
images; 

classifying said changes in position and shape of said animal as postures; and 
characterizing activity of said animal as one of a set of predetermined behaviors based on 

a comparison of a sequence of said postures to pre-trained models or rules of said 

set of predetermined behaviors. 

9. The method of claim 8, wherein said step of characterizing activity includes the steps of: 
describing said sequence of said postures as behavior primitives; and 

aggregating said behavior primitives into actual behavior over a range of images. 

1 0. The method of claim 9, wherein said step of characterizing activity further includes the 
steps of: 

describing a set of conditions and rules required for characterizing said activity; and 
matching and testing generated features to see if said condition and rules are satisfied. 

11. A method of determining and characterizing activity of an animal using computer 
processing of video images, comprising the steps of: 

detecting an animal in said video images; 
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tracking changes to said animal over a plurality of video images; 
identifying and classifying said changes to said animal; and 

characterizing said activity of said animal based on a comparison to pre-trained models or 
rules of such activity, wherein the step of detecting an animal in said video 
images includes 

applying a lenient threshold on a difference between a current image and a 
background so as to determine a broad region of interest; 

classifying by intensity values various pixels in said region of interest to obtain 

said animal, by selecting only those intensity values that belong to a set of 
model intensity values of said animal; and 

refining contours of said animal image by smoothing. 

1 2. The method of claim 8, wherein said step of classifying said changes in position and 
shape of said animal as postures includes using statistical and contour-based shape information, 

13. A method of determining and characterizing activity of an animal using computer 
processing of video images, comprising the steps of: 

detecting an animal in said video images; 

tracking changes to said animal over a plurality of video images; 

identifying and classifying said changes to said animal; and 

characterizing said activity of said animal based on a comparison to pre-trained models or 
rules of such activity, 
wherein said step of characterizing said activity includes the steps of: 
describing a sequence of postures as behavior primitives; 
aggregating behavior primitives into actual behavior over a range of images; 
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describing a set of conditions and rules required for characterizing said activities; and 
matching and testing generated features to see if said conditions and rules are satisfied; 

wherein said posture determination and description includes using statistical and contour-based 

shape information; 

wherein said step of identifying and classifying changes to said animal includes using statistical 
shape information selected from the group consisting of: 

area of said animal; 

centroid position of said animal; 

bounding box and aspect ratio of said bounding box of said animal; 
eccentricity of said animal; and 

directional orientation of said animal relative to an axis as generated with a Principal 
Component Analysis. 

14. The method of claim 12, wherein said step of classifying said changes in position and 
shape of said animal as postures uses contour-based shape information selected from the group 
consisting of curvature measures, thickness measures, relative orientation measures, length 
measures, and comer points. 

1 5. The method of claim 12, wherein said step of classifying said changes in position and 
shape of said animal as postures includes identifying a set of model postures and description 
information for said set of model postures, said set of model postures including a horizontal side 
view posture, a vertical posture, a cuddled posture, a horizontal front/back view posture, a 
partially reared posture, a stretched posture, a hang vertical posture, a hang cuddled posture, an 
eating posture, and a drinking posture. 
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16. The method of claim 12, wherein said step of classifying said changes in position and 
shape of said animal as postures includes classifying the statistical and contour-based shape 
information from a current image to assign a best-matched posture. 

17. The method of claim 9, wherein the said step of describing said sequence of said postures 
as behavior primitives includes identifying patterns of postures over a sequence of images, 

18. A method of determining and characterizing activity of an animal using computer 
processing of video images, comprising the steps of: 

detecting an animal in said video images; 

tracking changes to said animal over a plurality of video images; 

identifying and classifying said changes to said animal; and 

characterizing said activity of said animal based on a comparison to pre-trained models or 
rules of such activity, 
wherein said step of characterizing said activity includes the steps of: 

describing a sequence of postures as behavior primitives; 

aggregating behavior primitives into actual behavior over a range of images; 

describing a set of conditions and rules required for characterizing said activities; and 

matching and testing generated features to see if said conditions and rules are satisfied; 
wherein the said step of describing said behavior primitives includes the step of identifying 
patterns of postures over a sequence of images; and 

wherein said step of describing said behavior primitives step further includes the step of 
analyzing temporal information selected from the group consisting of direction and magnitude of 
movement of the centroid, increase and decrease of the eccentricity, increase and decrease of the 
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area, increase and decrease of the aspect ratio of a bounding box, and change in contour 
information. 

19. The method of claim 10, wherein the said step of aggregating said behavior primitives 
includes analyzing temporal ordering of said behavior primitives, such as using information 
about a transition from a previous behavior primitive to a next behavior primitive, and applying 

all applicable conditions and rules. 

20. The method of claim 19, wherein said analyzing temporal ordering of said behavior 
primitives is a time-series analysis such as Hidden Markov Model (HMM). 

2 1 . The method of claim 8, wherein said set of predetermined behaviors corresponds to a set 
of pre-trained behavior models. 

22. The method of claim 8, wherein said set of predetermined behaviors includes rearing up 
to a fiilly reared up or partially reared up position, which is determined by a sequence of postures 
starting from cuddled, horizontal side-view, or horizontal front/back view postures to ending in a 
vertical or partially reared posture. 

23. The method of claim 8, wherein said set of predetermined behaviors includes coming 
down from a reared up or partially reared up position, which is determined by a sequence of 
postures starting from vertical or partially reared postures to ending in a cuddled, horizontal side 
view or horizontal front/back view posture. 
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24. The method of claim 8, wherein said set of predetermined behaviors includes eating, 
which is determined by a sequence of eating postures where the mouth of said animal is in touch 
with a food container. 

25. The method of claim 8, wherein said set of predetermined behaviors includes drinking, 
which is determined by a sequence of drinking postures where the mouth of said animal is in 

touch with a water spoutt^ 

26. The method of claim 8, wherein said set of predetermined behaviors includes digging, 
which is determined by the aft movement of bedding by said animal with its fore and hind limbs. 

27. The method of claim 8, wherein said set of predetermined behaviors includes foraging, 
which is determined by the movement of bedding using the mouth and forelimbs. 

28. The method of claim 8, wherein said set of predetermined behaviors includes jumping, 

which is determined by a single up and down movement of said animal. 

29. The method of claim 8, wherein said set of predetermined behaviors includes jumping 
repetitively, which is determined by several continuous up and down movement of said animal, 

30. The method of claim 8, wherein said set of predetermined behaviors includes sniffing, 
which is determined by random brisk movement of the head while the rest of the body remains 
stationary. 
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3 1 . The method of claim 8, wherein said set of predetermined behaviors includes hanging 
from the top of the cage, which is determined by a sequence of postures starting from vertical 
posture to ending in a hang vertical or hang cuddled posture, 

32. The method of claim 8, wherein said set of predetermined behaviors includes landing 
after hanging, which is determined by a sequence of postures starting from a hang vertical or 
hang cuddled posture to ending in a vertical posture. 

33. The method of claim 8, wherein said set of predetermined behaviors includes sleeping, 
which is determined by the absence of major movements of the contour of said animal for a 
prolonged period of time. 

34. The method of claim 8, wherein said set of predetermined behaviors includes twitching 
during sleep, which is determined by detection of a brief period of substantial movement and 
then resumption of sleep activity. 

35. The method of claim 8, wherein said set of predetermined behaviors includes awakening 

from sleep, which is determined by a prolonged movement of said animal after sleep has set in. 

36. The method of claim 8, wherein said set of predetermined behaviors includes grooming, 
which is determined by brisk movement of limbs and mouth in a cyclical and periodic pattern. 

37. The method of claim 8, wherein said set of predetermined behaviors includes pausing 
briefly, which is determined by brief absence of movement of said animal. 
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38. The method of claim 8, wherein said group of behavior models includes the behavior of 
urinating, and said urinate behavior is determined by the detection of the tail being raised up and 
the animal remaining stationary briefly. 

39. The method of claim 8, wherein said set of predetermined behaviors includes turning, 
which is determined by a sequence of postures starting from horizontal side view or cuddled 

posture to ending in a horizontal front/back view posture, and vice versa. 

40. The method of claim 8, wherein said set of predetermined behaviors includes circling, 
which is determined by three or more successive turns. 

41 . The method of claim 8, wherein said set of predetermined behaviors includes walking or 
running, which is determined by the continuous sideways movement of the centroid of said 
animal. 

42. The method of claim 8, wherein said set of predetermined behaviors includes body 
stretching vertically or horizontally, which is determined by a concave shape of said animal's 
back. 

43. The method of claim 8, wherein said set of predetermined behaviors includes chewing, 
which is determined by the movement of the mouth while the mouth is not in touch with a food 
container. 
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44. The method of claim 8, wherein said set of predetermined behaviors includes remaining 
stationary, which is determined by said animal remaining in the same place and not performing 
any of the other predetermined behaviors. 

45. The method of claim 8, wherein said set of predetermined behaviors includes unknown 
behavior, which is activity that cannot be characterized by any of a set of behavior models. 

46. The method of claim 8, wherein said steps are performed in night conditions by using red 
light to simulate such night conditions, or by using infra-red cameras to capture images with no 
Ught. 

47. The method of claim 8, wherein said steps are performed with a plurality of cages or 
arenas, each of which contains a single animal. 

48. The method of claim 8, wherein said step of detecting an animal includes detecting body 

parts of said animal. 

49. The metiiod of claim 48, wherein said body parts include the head. 

50. The method of claim 48, wherein said body parts include the tail. 

5 1 . The method of claim 48, wherein said body parts include the ear. 

52. The method of claim 48, wherein said body parts include the upper and lower back. 
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53. The method of claim 48, wherein said body parts include the abdomen. 



54. The method of claim 48, wherein said body parts include the hind-limbs. 



55. The method of claim 48, wherein said body parts include the forelimbs. 
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EVIDENCE APPENDIX 



None. 
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RELATED PROCEEDINGS APPENDIX 
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